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DETAILED ACTION 



1. This Office action for U.S. Patent Application 10/713,130 is responsive to the 
Request for Continued Examination filed 03 December 2008, in reply to the Final 
Rejection of 15 September 2008 and the Advisory Action of 17 November 2008. 
Currently, claims 11, 12, 14-17, 19-24, 35, 49-53, 55-58, 60-65, and 66-70 are 
pending. 

2. In the Final Rejection of 15 September 2008, the specification was objected to for 
an informality. Claims 11, 12, 14-17, 19, 20, 23, 24, 35, 49-53, 55-58, 60, 61, 61, 64, 
65, and 67-70 were rejected under 35 U.S.C. 103(a) as obvious over U.S. Patent 
6,111,604 A (Hashimoto et al.) in view of US Patent 6,148,031 A (Kato) and in view of 
ISO/IEC 11172-1 (MPEG-1). Claims 21, 22, 62, and 63 were rejected under 35 U.S.C. 
103(a) as obvious over Hashimoto et al. in view of Kato and MPEG-1 and in view of US 
6,327,423 61 (Ejima etal.). 



Continued Examination Under 37 CFR 1.114 

3. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03 
December 2008 has been entered. 
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Response to Arguments 

4. Applicant's arguments with respect to claim 11 have been considered but are 
moot in view of the new ground(s) of rejection. US Patent 5,987,179 A (Riek et al.) 
explicitly discloses a video encoder in which still picture data is integrated within a 
moving picture image sequence, and in which still picture data may be encoded as 
"enhanced" I pictures, P pictures, or B pictures. The Hashimoto et al. reference, 
previously relied on for this limitation in the 17 November 2008 Advisory Action, is more 
ambiguous as to the claimed limitation of still picture data in a pack containing I picture 
data as well as P or B picture data. 

Claim Rejections - 35 USC § 101 

5. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

6. Claims 52, 53, 55-58, 62, 63, 67, 68, and 70 are rejected under 35 U.S.C. 101 
because the claimed invention is directed to non-statutory subject matter. Supreme 
Court precedent^ and recent Federal Circuit decisions^ indicate that a statutory 
"process" under 35 U.S.C. 101 must (1) be tied to another statutory category (such as a 
particular apparatus), or (2) transform underlying subject matter (such as an article or 
material) to a different state or thing. While the instant claim(s) recite a series of steps 

' Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Flook, 437 U.S. 584, 588 n.9 (1978); Gottschalk v. Benson, 
409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 US 780, 787-88 (1876). 
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or acts to be performed, the claims neitlier transform underlying subject matter nor 
positively tie to another statutory category that accomplishes the claimed method steps, 
and therefore do not qualify as a statutory process. The above claims do not recite 
what apparatus performs the claimed encoding steps. The "photographing means" and 
"audio inputting means" in claim 52, for example, are not recited as performing any 
active steps in the claimed method. Contrast this with claims 61, 64, and 65, which all 
tie method steps to specific devices such as the "record medium" of claim 61, the 
"picture decoding means" of claim 64, described as performing the claimed step of 
decoding the stored encoded picture data, or the "photographing means" of claim 65, 
described as performing the claimed step of outputting a picture signal. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 11, 12, 14-17, 19, 20, 23, 24, 35, 49-53, 55-58, 60, 61, 64, 65, and 67- 
70 are rejected under 35 U.S.C. 103(a) as being unpatentable over US Patent 
6,111,604 A (Hashimoto et al.) in view of US Patent 6,148,031 A (Kato), in view of US 
Patent 5,987,179 A (Riek et al.), and in view of ISO/IEC 11172-1 (MPEG-1 Part 1). 
Hashimoto et al. teaches a digital camera. 



^ In reBilski, 88 USPQ2d 1385 (Fed. Cir. 2008). 
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Regarding claims 11, 35, and 52, figure 8 of Hasliimoto et al. shows a block 
diagram of the camera. Image photographing section 6 comprises lens 7, lens opening 
8, imaging element 9, and filter 10. The analog input image signal is converted in 
analog/digital converter 4 and further processed in DSP 11 (column 6: lines 40-61). 
This corresponds with the claimed "photographing means". Audio signals are input into 
microphone 1 and output through amplifier/filter 2a to analog/digital converter 4 (column 
6: lines 18-26). This corresponds with the claimed "audio inputting means". Image data 
compression/expansion circuit 12 encodes the images from DSP 1 1 in a format such as 
JPEG or MPEG (column 6: line 62-column 7: line 2). This corresponds with the 
claimed "video encoding means" that performs the steps of "encoding the video signal" 
in the two encoding methods in claims 35 and 52. Figure 11 of Hashimoto et al. 
illustrates the process for capturing video and information. When the user presses the 
shutter button, a first picture with associated audio is captured. Image and audio files 
are stored in memory card 16, and an association file is written to link the image and 
audio files together (column 9: lines 46-54). The association file may be a container file 
for a still image, a still image with audio data, or a moving image with audio data 
(column 10: lines 1-8). 

The present invention differs from Hashimoto et al. in that in the present 
invention, two encoding methods exist: a first mode encoding only still image data, and 
a second mode encoding audio data with still picture data or moving picture data 
comprising intra I frames and inter P or B frames. However, in Hashimoto et al., no 
clear distinction is made between various encoding modes for the recorded pictures. In 
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the example given in column 9: line 55-column 10: line 4, audio may be associated with 
a still JPEG image, or a moving MJPEG image, or an MPEG image (column 6: line 65), 
which was known in the art at the time the invention was made to incorporate sound 

data. 

Kato teaches an image processing system in a digital camera. Regarding claims 
1 1 and 35, in Kato, in a continuous imaging mode, input images are initially recorded in 
real time in an intra mode as a succession of JPEG images, and stored in memory 20 
(column 3: lines 41-47). In a still image mode, the input image is recorded in memory 
20 as a single JPEG image (column 3: lines 47-53). After recording is finished, system 
control circuit 26 re-encodes the recorded series of intra images in an inter-frame 
compression mode (column 3: lines 54-63). This system control circuit corresponds 
with the claimed "controlling means" of claim 1 1 , and the selection of a still image mode 
or a motion image mode in Kato corresponds with the claimed recording mode selection 
in claims 35 and 52. 

Hashimoto et al. discloses a portion of the claimed invention, but not encoding 
pictures according to two different encoding methods. Kato teaches that it was known 
to encode motion image data in a separate format than still image data. Therefore, it 
would have been obvious to one having ordinary skill in the art at the time the invention 
was made to modify the camera of Hashimoto et al. to re-encode pictures having a 
temporal aspect, such as pictures with associated sound, as inter-frame encoded 
images after encoding, as taught by Kato, since Kato states in column 2: lines 16-34 
that such a modification would enable the final recording to be achieved with higher 
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compression than witli intra pictures alone, wliile maintaining the ability for a user to 
record a high-quality still image during the motion image recording process. 

The present invention differs from the combination of Hashimoto et al. and Kato 
in that in the present invention, a still image recorded with sound is recorded in the 
same mode containing I pictures, P pictures, or B pictures as motion images recorded 
with sound, in Hashimoto et al., a still picture recorded with sound is a single JPEG 
image with an associated audio file. 

Riek et al. discloses a camera that encodes still images in an MPEG bitstream. 
Regarding claim 1 , figure 2 illustrates an embodiment of the Riek et al. apparatus. Light 
is input through lens 12 to CCD 14, which forms images (column 4: lines 15-18). These 
images are converted to a standard digital format in ISO CCIR601 converter 27 (column 
4: lines 35-38). As will be shown below, the images received may be encoded as still 
images or moving images. A user may switch from recording motion images to 
recording still images with still select button 22 which causes logic and control unit 32 to 
encode a still image (column 4: lines 41-50). During a still image mode, a still image 
stored in frame store 29 from converter 27 is selected for encoding (column 4: lines 41- 
46), rather than directly from the converter 27. Encoder 30 encodes a still image as a 
series of zero-motion-vector B frames or an enhanced I frame or P frame followed by a 
series of B frames, and encoding the first frame at the conclusion of recording the still 
image as the next I frame (column 9: line 22-column 10: line 41). Then, Riek et al. 
discloses the claimed second encoding method which may capture both still picture 
data and moving picture data, each comprising I pictures and P pictures or B pictures. 
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Hashimoto et al., in combination with Kato, discloses a majority of the claimed 
invention except for a single encoding method suitable for both still and motion image 
data. Riek et al. teaches that it was known to encode a sequence of still pictures as 
MPEG pictures integrated within a motion picture sequence. Therefore, it would have 
been obvious to one having ordinary skill in the art at the time of the invention to modify 
the encoder of Kato to record still pictures containing sound as inter pictures as with 
motion pictures containing sound, as taught by Riek et al., since Riek et al. states in 
column 3: lines 1-59 that such a modification would allow for high quality still images to 
be encoded within a motion image at a relatively low bit rate taking advantage of the 
inter-picture coding techniques of MPEG. 

However, while the present invention is directed to multiplexing an encoded 
picture signal and an encoded audio signal, Hashimoto et al. does not give details of its 
process of "combining" a video file and an audio file (column 1 1 : lines 34-42). 

MPEG-1 Part 1 defines the system coding layer of an MPEG-1 coded data 
stream, in which audio and video data streams are multiplexed (forward). Regarding 
claims 11, 35, and 52, Section 1-A.6.3 illustrates a sample multiplexing of a stream 
having one video and one audio stream. The stream is divided into packs, each of 
which has a header and three packets, each of 2048 bytes. First, 13 video packets are 
transmitted to ensure successful buffering. Then, an audio packet is placed for every 
6.25 video packets. Section 1-A.6.9 shows an extended sample multiplexed data 
stream. Here, a second audio packet is placed between the twentieth and twenty-first 
video packets. However, while in the shown example, one audio packet is placed for 
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multiple video packets, the examiner takes Official Notice that it was known in the art for 
audio and video packets to be correlated in a 1 :1 ratio, as in the "locked audio" of DV, in 
which one audio pack is present for each frame. Such a modification would ease linear 
editing to prevent mismatches between audio and video data streams at a start point or 
an end point of an edited segment. 

Hashimoto et al., in combination with Kato and Riek et al., discloses the claimed 
invention except for multiplexing an audio and picture signal. MPEG-1 Part 1 teaches 
that it was known to produce a multimedia datastream by multiplexing packets of audio 
and video data. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made to implement the combination of a video and 
audio file in Hashimoto et al. as a multiplexing operation, as taught by MPEG-1 Part 1, 
since MPEG-1 Part 1 states in the Introduction that such a modification would allow for 
synchronized playback of audio and video data without having to buffer an entire sub- 
stream. 

Regarding claims 12 and 53, in Kato, "the JPEG standard is used in the still 
image compression and the MPEG standard is used in the moving image compression" 
(column 9: lines 12-14). 

Regarding claims 14 and 55, figure 12 of Hashimoto et al. shows video files and 
audio files stored in separate areas of memory card 16. 

Regarding claims 15 and 56, in Kato, as mentioned previously, video data is first 
stored in first memory 20 and then transferred to second memory 22 (column 3: lines 
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54-63). This corresponds witli writing multiplexed data to memory, reading the 
multiplexed data from memory, and recording multiplexed data on a recording medium. 
Additionally, in Kato, during the recording of a moving image, a still image from the 
sequence of moving images may be additionally transferred from the first memory to the 
second memory as an intra picture in an independent process of the moving picture 
recording (column 4: lines 1-10). This corresponds with encoding a video signal in the 
"first" encoding method, writing the signal to the memory, reading the signal from the 
memory, and recording the signal to the recording medium. 

Regarding claims 16 and 57, figure 5 of Kato shows a compressed video 
encoder including DCT circuit 107, quantizing circuit 108, and variable length coding 
circuit 115 (column 5: line 60-column 6: line 24). 

Regarding claims 17 and 58, figure 12 of Hashimoto et al. illustrates audio and 
video files stored in the memory as having headers. 

Regarding claims 19 and 60, an MPEG-1 pack, containing 3 packets, is designed 
to have a pack rate of 29 Hz, or 1 frame per pack (Section 1-A.6.3). 

Regarding claims 20 and 61, Kato temporarily stores incoming data on first 
memory 20, and after re-encoding, permanently stores the data on second memory 22 
(column 4: lines 45-55, "the second memory 22 is the final storage medium"). 

Regarding claims 23 and 64, in Hashimoto et al., image data compression circuit 
12 may also perform image decoding (column 6: lines 62-66), and so corresponds with 
the claimed "video decoding means". The decoded video signals may be further 
processed in DSP 11 (column 6: lines 58-61), which outputs a video signal 26 to a 
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display such as an LCD viewfinder (not sliown in figure 8). Tliis display corresponds 
with the claimed "displaying means". Digital audio signals may also be decoded in 
audio data compression/expansion circuit, transmitted to D/A converter, amplified and 
filtered in amplifier 2b, and output in output stream 26 to speaker 32 (column 5: lines 17- 
39). This corresponds with the claimed "audio outputting means". This process of 
reading data stored in memory card 16 (column 7: lines 34-50), like all other processes 
of the camera of Hashimoto et al., is controlled by CPU 23 (column 7: lines 15-16), 
which corresponds with the claimed "controlling means". 

Regarding claims 24 and 65, the CCD in Hashimoto et al. has a resolution of 768 
X 480 pixels (column 6: line 44), and Kato inputs images at a resolution of 720 x 480 
pixels, in accordance with the NTSC standard (column 4: line 15), and produces an 
output of 320 X 240 pixels (column 4: line 24), in accordance with the GIF format. 
Although neither Hashimoto et al. nor Kato et al. record pictures at the VGA 640 x 480 
pixel standard, it would have been an obvious matter of design choice to modify the 
image sensing portion of camera of Hashimoto et al. or of Kato to produce 640 x 480 
pictures, since it has been held that a change in size of a component is generally 
recognized as being within the level of ordinary skill in the art. See In re Rose, 105 
USPQ 237 (CCPA 1955). 

Regarding claim 49, in Kato, as shown in figure 1, system control circuit 26 
controls both the image compression circuit 18, first memory 20, and second memory 
22 (column 3: lines 41-63). This system control circuit corresponds with the claimed 
"controlling means". 
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Regarding claims 50 and 67, in Hashimoto et a!., incoming image data from a 
camera is processed in noise reduction circuit 10 and DSP 11 (column 6: lines 40-61), 
and incoming audio data from a microphone is processed in amplifier/filter 3a (column 
6: lines 18-21). 

Regarding claims 51 and 68, in Hashimoto et al., figure 14 illustrates the 
flowchart for transmitting and receiving data from the camera to an external device 
(column 10: line 41-column 11: line 42). Data from the memory card is transferred to 
FIFO 13 (column 11: lines 25-29), and transmitted to an external device via interface 
circuit 27 (column 7: lines 1-36). Like every other process in the camera of Hashimoto 
et al., this process is controlled by CPU 23 (column 7: lines 15-16), which corresponds 
with the claimed "controlling means". 

Regarding claims 69 and 70, in Kato, digital signal processor circuit 14 
corresponds with the claimed "first picture encoder" that performs the step of receiving a 
picture signal. In Hashimoto et al., audio data compression circuit 3 corresponds with 
the claimed "second encoder" that performs the step of receiving an audio signal. In 
Kato, image compression circuit 18, as modified by Riek et al., corresponds with the 
claimed "picture generation" that performs the claimed step of "generating fixed data". 
In Hashimoto et al., FIFO 13, which combines audio files and image files (column 11: 
lines 43-61), corresponds with the claimed "third encoder" that performs the claimed 
step of "multiplexing". 
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9. Claims 21, 22, 62, and 63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hashimoto et al. in view of Kato, Riek et al., and MPEG-1 as applied 
to claims 1 1 and 52 above, and further in view of US Patent 6,327,423 B1 (Ejima et al.)- 
Claims 21, 22, 62, and 63 are directed to specific operations of causing a camera to 
perform an audio capture for a certain time. Hashimoto et al. teaches taking pictures 
when a shutter button is pressed (column 7: lines 20-24), Kato teaches operating a 
keyboard to issue image taking commands (column 3: lines 41-53), and Riek et al. 
teaches recording a still image while a still select button is depressed (column 4: lines 
41-50). However, the above references do not teach operation for a time period to 
encode audio data. 

Ejima et al. teaches a camcorder that records sound data. Regarding claims 21 
and 62, figure 14 is a flowchart illustrating one embodiment of the sound recording 
control process of Ejima et al. At step SI, CPU 39 determines if a release switch 10 is 
pressed, and if it is, the image recording process begins at step S2 (column 15: line 64- 
column 16: line 3). At step S3, the sound recording process is started, and at step S4, a 
"REC" display is shown on a viewfinder to indicate that sound is being recorded (column 
16: lines 4-11). At step S5, after 10 seconds have passed, the sound recording process 
stops (column 16: lines 11-16, 34-40). However, if a sound recording switch is pressed 
within 10 seconds at step S6, sound recording continues (column 16: lines 14-23, 44- 
50). The sound recording then ends when the sound recording switch is released at 
step S20 (column 16: lines 23-50). Then, sound recording switch 12 corresponds with 
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the claimed "operating means", and the time period in which the sound recording switch 
is pressed corresponds with the claimed "timing means". 

Hashimoto et al., in combination with Kato, Riek et al., and MPEG-1, discloses 
the claimed invention except for encoding audio during the pressing of an operation 
means. Ejima et al. teaches that it was known to perform sound recording while a 
sound recording switch is pressed. Therefore, it would have been obvious to one 
having ordinary skill in the art at the time the invention was made to incorporate the 
sound recording switch of Ejima et al. into the camera of Hashimoto et al. or Kato, since 
Ejima et al. states in column 1 : line 60-column 2: line 20 that such a modification would 
allow the timing of a sound recording to be independent of the timing of its associated 
video recording. 

Regarding claims 22 and 63, in Kato, if sound recording switch 12 is not pressed, 
then release switch 10 corresponds with the claimed "operating means", and the ten 
seconds is the "predetermined time period" in which audio is encoded. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
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number for the organization wliere tliis application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

ID. N. W./ 

Examiner, Art Unit 2621 



/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 



